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Abstract 

The  Center  for  the  Management  of  Information  (CMI) 
at  The  University  of  Arizona  engaged  in  a  joint  research 
project  with  the  U.S.  Navy’s  Commander  Third  Fleet 
(Third  Fleet)  and  The  MITRE  Corporation  (MITRE)  to 
use  and  evaluate  collaborative  technology  during  Strong 
Angel,  a  humanitarian  assistance/disaster  relief 
(HA/DR)  exercise.  Strong  Angel  was  a  part  of  RIMPAC 
2000,  a  five-week  multinational  exercise  that  involved 
seven  nations  with  over  22,000  people,  fifty  ships,  and 
200  aircraft.  RIMPAC  2000’ s  Strong  Angel  set  out  to 
satisfy  three  goals:  (1)  Develop  a  mutual  understanding 
of  respective  capabilities,  limitations  and  expectations 
among  multinational  militaries  and  the  main  United 
Nations  relief  agencies;  (2)  Create  a  replicable  system 
for  the  safe  conduct  of  Strong  Angel  and  subsequent 
exercises  in  civil-military  interaction  for  humanitarian 
support;  and  (3)  Deliver  a  coordinated  response  to  a 
population  in  crisis.  CMI,  Third  Fleet,  and  MITRE 
teamed  to  achieve  four  objectives:  (1)  provide  a 
collaborative  environment  both  at  sea  and  ashore  within 
an  austere  environment;  (2)  use  collaborative 
technology  to  establish  a  forum  for  the  exchange  of 
relevant  information  between  civilian  humanitarian 
organizations  and  the  military;  (3)  document  the  flux  of 
combined  activities  each  day;  and  (4)  evaluate  the  utility 
of  collaborative  technology  during  a  civil-military 
exercise  in  humanitarian  relief.  The  team  met  each 
objective  and  reports  the  results  in  this  paper. 

CMI,  MITRE,  &  Third  Fleet 

In  1996,  CMI  partnered  with  Third  Fleet  to  develop 
collaborative  processes  and  technologies  to  overcome 
the  limitations  of  performing  closely  coordinated  staff 


work  aboard  a  U.S.  Navy  ship.  The  Third  Fleet  staff  is 
required  to  collaborate  with  hundreds  of  people 
internally  and  across  militaries,  civilian  organizations, 
and  governments  during  daily  operations  and  in  crisis 
situations.  Collaboration  across  time  and  space  occurs 
regularly  at  Third  Fleet,  so  the  opportunity  to  research 
the  impact  of  collaborative  technologies  and  processes  is 
ideal.  The  Third  Fleet  Commander  is  a  Vice  Admiral 
(three  stars)  and  is  stationed  aboard  the  USS  Coronado 
in  San  Diego,  California.  A  detailed  description  of  the 
mission  for  Third  Fleet  is  povided  in  Adkins,  et  al., 
(2000). 

CMI  has  a  long  history  of  creating  and  fielding  robust 
collaborative  application  prototypes  and  corresponding 
facilitation  methods  to  enhance  the  performance  of 
teams  working  toward  a  decision  or  common  goal.  The 
result  of  that  work  is  now  embodied  in  GroupSystems,  a 
suite  of  collaborative  software  tools.  As  the  nature  and 
composition  of  organizational  teams  have  changed  in  the 
last  few  years,  CMI  has  begun  to  research  opportunities 
for  distributed  collaboration. 

This  research  has  led  to  the  development  of  a  suite  of 
collaborative  tools  that  are  designed  to  operate  over  the 
Internet  and/or  an  intranet.  These  tools  allow  teams  to 
collaborate  on  various  types  of  projects  in  several 
different  settings:  same -time/same -place,  same- 
time/different-place,  and  different-time/different -place. 
Distributed  teamwork  has  also  presented  new  challenges 
in  meeting  dynamics,  which  is  also  part  of  the  current 
CMI  research  focus. 

The  Information  Technology  Center  at  MITRE  brings 
extensive  experience  in  developing  collaboration 
environments  and  evaluation  methodologies  for 
collaborative  technologies  and  tools  (Drury,  et.  al., 
1999).  Its  computer-human  interaction  (CHI)  research 
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focus  includes  usability  studies,  speech  and  natural 
language  interfaces,  and  intelligent  information  access. 
One  specific  strength  is  hands-on  experience  with  the 
integration  of  technologies  and  tasks,  another  is 
developing  techniques  for  capturing  and  analyzing  data 
on  tool  usage  and  collaborative  interaction. 

MITRE  is  a  not-for-profit  corporation  working  in  the 
public  interest,  in  partnership  with  government.  It 
addresses  issues  of  critical  national  importance, 
combining  systems  engineering  and  information 
technology  to  develop  innovative  solutions.  MITRE 
operates  through  three  Federally  Funded  Research  and 
Development  Centers,  focusing  on  the  Department  of 
Defense,  the  FAA  and  civil  aviation,  and  the  IRS  and 
Treasury  Department. 

Description  of  Strong  Angel  Exercise 

The  Strong  Angel  exercise  was  developed  in  response 
to  changes  in  the  requirements  for  national  defense. 
According  to  Kaplan  (1996),  VanCreveld  (1991)  and 
others,  since  1989  the  world  has  undergone  a 
transformation  that  seems  destined  to  remake  the 
understanding  of  both  requirements  for  domestic  safety 
and  sense  of  “noblesse  oblige.  ”  With  the  fall  of  the  only 
other  superpower,  the  international  community  has 
watched  the  increasing  number  of  small-scale  conflicts 
throughout  the  world  and  has  glimpsed  environmental 
pressures  beginning  to  re -define  war.  The  national 
leadership  has  struggled  to  find  an  appropriate  response, 
with  variable  success,  occasionally  deciding  that  active 
intervention  in  remote  conflicts  serves  our  national 
interest.  Unfortunately,  the  military  does  not  effectively 
train  for  such  action,  and  so  each  intervention  has 
required  the  re -learning  of  hard  lessons. 

Current  military  thinking  was  predominantly  forged  in 
the  crucible  of  World  War  II.  Military  strategies  were 
later  shaped  by  the  Cold  War  and  the  United  States 
success  seemed  to  prove  the  worth  of  the  high  intensity, 
force  on  force  perspective.  The  outcomes  in  Korea  and 
Vietnam  were  far  less  clear  and  gave  a  hint  of  what  was 
to  come. 

The  military  now  realizes  that,  with  the  overarching 
power  struggle  gone,  so  is  much  of  the  world's  former 
stability.  Small,  brutal,  internecine  conflicts,  reminiscent 
of  Korea  and  Vietnam,  have  come  to  dominate  the  world 
landscape  with  violence  that  often  now  falls  squarely  on 
the  backs  of  the  innocent.  Civilians  have  been  starved  to 
clear  a  disputed  border  area,  women  raped  to  instill 
terror  and  a  religious  ostracism,  the  elderly  singled  out 
by  snipers  for  the  psychological  impact  of  the  brutality, 
and  children  mutilated  to  consolidate  fear  and 
submissive  compliance.  In  the  face  of  such  barbarity. 


America  often  perceives  a  moral  obligation  to  intervene. 
In  addition,  these  smaller  conflicts  pose  a  risk  to  global 
stability  on  which  trade  relationships  depend,  and  so,  for 
various  reasons,  U.S.  national  interests  are  involved.  As 
the  military  has  responded,  leaders  have  recognized  the 
military  does  not  possess  all  skills  required  to  be 
effective  in  these  new  environments. 

Strong  Angel  was  designed  to  force  the  development 
of  a  few  required  skills.  The  military  challenged 
themselves  to  respond  helpfully  to  international  agencies 
that  routinely  and  effectively  manage  populations -in- 
crisis.  The  team  formed  exercise  objectives  to  ensure  all 
involved  would  have  to  leam  from  those  more 
experienced.  Real-world  events  were  used  to  establish  a 
valid  scenario  for  the  exercise. 

The  scenario  had  two  nations  engaged  in  a  low- 
intensity  conflict,  with  harassment  of  a  minority 
population  within  the  aggressor  country.  Harassment 
quickly  became  violent  persecution,  and  thousands  of 
refugees  then  escaped  across  a  border.  The  country  that 
received  the  refugees  was  rapidly  overwhelmed  and 
asked  for  help  in  refugee  management.  The  UN  Security 
Council  passed  a  resolution  requesting  international 
assistance  and  a  Task  Force  of  ships  from  seven  nations 
was  ordered  to  respond.  Multi-national  military 
resources  were  placed  in  support  of  UN  relief  agencies, 
and  a  refugee  camp  was  built. 

The  exercise.  Strong  Angel,  was  held  in  early  summer 
of  2000  on  volcanic  dust  in  the  northwest  corner  of  the 
Big  Island  of  Hawaii  near  the  town  of  Waimea.  With  the 
bulk  of  the  military  presence  initially  in  Honolulu,  the 
UN  agency  participants  came  to  Pearl  Harbor  and 
walked  onto  the  USS  Coronado,  Command  Ship  for 
Third  Fleet,  almost  as  soon  as  the  mock  Security  Council 
Resolution  was  passed.  As  the  refugee  camp  material 
began  to  be  loaded  onto  amphibious  ships  anchored  in 
Pearl  Harbor,  the  USS  Coronado  set  sail  for  the  Big 
Island.  Meetings  of  the  Humanitarian  Planning  Group 
aboard  USS  Coronado,  at  sea,  then  began  in  earnest. 

One  hundred  and  twenty-five  American  Red  Cross 
Disaster  Services  volunteers  served  as  the  refugee 
population.  As  the  Civil-Military  Operations  Center 
(CMOC)  took  shape  afloat,  preparations  to  establish  a 
refugee  camp  ashore  revealed  a  number  of  unfamiliar 
details  to  the  UN  and  military  organizations.  Only  36 
hours  had  been  scheduled  to  design  and  establish  the 
camp  once  the  UN  and  military  arrived  ashore.  The 
refugees  began  arriving  at  about  the  36-hour  mark,  very 
late  in  the  evening.  The  consequent  urgency  lent  a  sense 
of  realism  to  the  problem  because  real  civilians  were 
arriving  by  truck  into  the  middle  of  nowhere  at  night, 
with  no  water,  power,  sanitation,  or  shelter  unless  all 
cooperating  agencies  succeeded  in  their  obligations.  For 
additional  drama,  the  Exercise  Control  Group  planted 


actors  within  the  refugees  to  present  a  range  of  threats 
and  management  problems  as  the  refugees  arrived. 

The  exercise  ran  for  roughly  five  days  with  U.S. 
Marines  providing  almost  all  aspects  of  camp 
management  under  UN  direction.  At  the  completion  of 
that  week  the  consensus  across  all  participants,  including 
the  refugee  volunteers,  was  of  hard  won  success.  Strong 
Angel  has  since  been  reviewed  as  seminal  to  the 
development  of  exercises  in  humanitarian  support  and 
civil-military  integration  throughout  the  Pacific  Rim. 

Strong  Angel  Environmental  Conditions 

Austere  conditions  such  as  those  encountered  during 
Strong  Angel  dictate  heightened  precautions  and 
substantial  levels  of  maintenance  for  information 
technology  (IT)  equipment  and  supplies.  IT  equipment 
requires  protection  from  water,  dust  and  temperature 
extremes.  Electrical  power  is  often  unavailable  or 
unreliable.  Power  for  IT  in  this  environment  requires 
power  conditioners,  uninterruptible  power  supplies,  and 
battery  operation  where  practical,  such  as  with  laptops, 
handheld  computers  and  PDAs,  and  other  mobile  gear. 
Solar  panels  and  AC  inverters  operated  from  trucks  are 
expedient  backup  sources  of  electrical  power. 

Commercial  grade  equipment,  usually  intended  for  use 
in  office  environments,  is  particularly  susceptible  to 
damage  from  the  elements.  Makeshift  covers  and  filters 
can  be  utilized  to  protect  the  equipment;  and  compressed 
air  can  be  used  to  minimize  the  accumulation  of  dust 
inside  equipment.  However,  for  any  extended 
humanitarian  operation  it  will  be  critical  to  migrate  such 
equipment  into  appropriate  shelters  or  ventilated  cabinets 
to  prevent  contamination  by  water  and  dust;  and  to 
minimize  temperature  fluctuations. 

For  24x7  operations,  a  combination  power  conditioner 
and  UPS  capable  of  operating  the  telephone  and  IT 
network  infrastructure  for  several  hours  at  a  time  is 
critical.  Intolerance  by  commercial  grade  equipment  of 
substantial  over  or  under-voltage  conditions  necessitates 
the  use  of  a  power  conditioner  even  if  a  large  UPS  for 
long-term  backup  is  not  required.  A  number  of  new, 
scalable,  portable  hybrid  power  sources  such  as  solar 
charged  batteries  with  generator  backup  or  fuel  cells,  can 
provide  long-term,  reliable  electrical  power. 

Collaborative  Research  Context 

The  research  team’s  goal  in  a  broad  sense  was  to 
achieve  a  set  of  four  objectives  derived  from  the 
overarching  goals  of  the  Strong  Angel  exercise.  These 
objectives  were  to:  provide  a  collaborative  environment 
at  sea  and  ashore  in  an  austere  environment;  use 


collaborative  technology  to  establish  a  forum  to 
exchange  relevant  information  between  humanitarian 
organizations  and  the  military;  document  the  input  and 
output  of  HA/DR  activities  each  day;  and  evaluate  the 
utility  of  collaborative  technology  during  a  HA/DR 
exercise. 

The  team  devised  and  implemented  a  number  of 
workaround  field-expedient  shortcuts.  To  gain 
flexibility  aid  avoid  the  pitfalls  of  a  traditional  wired 
network,  the  research  team  implemented  a  wireless  LAN 
comprising  a  workstation  server  and  21  lightweight 
laptop  computers.  The  wireless  LAN  significantly  cut 
down  on  the  footprint  of  the  network  and  afforded  users 
the  opportunity  to  move  machines  around  easily  because 
they  were  able  to  access  the  wireless  LAN  anywhere  in 
camp.  The  server  was  also  a  proxy  server  connected 
through  a  T1  line  into  the  Internet.  CMI  and  Third  Fleet 
had  already  developed  appropriate  environments  aboard 
the  USS  Coronado  during  previous  research  programs 
(Briggs,  Mittleman,  Weinstein,  Nunamaker,  &  Adkins, 
1998). 

To  achieve  the  second  and  third  goals,  the  researchers 
used  two  software  applications.  One  was 
GroupSystems,  a  group  support  system  (GSS)  that  helps 
facilitate  group  interaction  (Nunamaker,  et.  al.,  1991). 
The  other  was  CommandNet,  a  real  time  collaborative 
log  used  to  help  the  group  maintain  situational 
awareness.  Finally,  the  research  team  evaluated  the  use 
of  the  collaborative  systems  during  the  exercise  to 
illuminate  issues  associated  with  the  development  and 
application  of  collaborative  technology  in  the  HA/DR 
context. 

GroupSystems  use  during  Strong  Angel 

The  team  learned  through  previous  sessions  with  the 
military  and  civilian  organizations  that  GSS  could  be 
useful  in  streamlining  and  coordinating  meetings  that 
took  place  during  the  exercise.  In  addition,  Nunamaker, 
et.  al.  (1991)  suggests  the  GSS  lowers  the  barriers  to 
communication  between  disparate  groups.  The 
researchers  hoped  to  build  trust  across  the  military  and 
civilian  organizations  and  further  improve  the  operation 
by  using  a  GSS.  This  process  was  more  complex  than 
was  anticipated. 

The  UN  team  was  comprised  of  field  operatives  from 
several  UN  agencies  located  around  the  world,  most  of 
whom  did  not  work  together  regularly  or  know  each 
other.  As  would  be  expected  each  subgroup  worked 
according  to  the  procedures  of  the  agency  they 
represented.  These  highly  skilled  professionals  were 
accustomed  to  being  dropped  into  austere  conditions 
with  little  notice  or  time  for  preparation.  The  UN  staff 


has  to  use  the  resources  at  their  immediate  disposal  to 
get  the  relief  operation  moving.  Often,  they  are 
subjected  to  physical  danger,  illness,  crime,  politics  and 
the  whims  of  the  media  and  popular  opinion.  As  a 
consequence,  the  UN  field  staff  is  extremely  self- 
sufficient  and  wary  of  outsiders. 

The  culture  and  mandate  of  the  UN  agencies  also 
stimulates  the  need  for  a  collective  voice.  As  a  political 
entity,  the  UN  and  its  agencies  must  be  sensitive  to  the 
positions  of  each  member  state,  the  local  population, 
local  politics,  the  host-nation  government, 
nongovernmental  organizations,  the  military  and  a  host 
of  other  players.  Through  experience,  UN  staff  have 
learned  that  an  effective  way  to  function  is  to  establish 
positions  in  private,  then  provide  outside  entities  with  a 
single  united  front.  This  approach,  though  sometimes 
cumbersome,  allows  UN  staff  to  navigate  the  charged 
political  landscape  while  still  maintaining  effectiveness. 
As  a  result  of  the  established  dynamics  of  not  publicly 
sharing  in- progress  ideas,,  the  use  of  a  GSS  by  the  UN 
contingent  was  an  issue.  However,  the  reasons  for  their 
hesitation  in  participating  were  not  yet  clear  to  the 
research  team. 

Aboard  the  USS  Coronado  the  research  team  was 
prepared  to  use  the  collaborative  tools  to  provide 
decision  support  and  to  document  the  lessons  learned  by 
the  military  and  the  UN.  At  the  preplanning  conference 
six  months  prior  to  the  exercise,  all  the  participants 
recognized  the  need  for  recording  each  day’s  activities, 
decisions  and  information  to  help  others  work  together  in 
future  exercises  and  actual  operations. 

A  lessons  -learned  session  was  scheduled  at  the  end  of 
each  day  to  allow  the  players  an  opportunity  to  distill 
and  record  important  ideas.  The  first  barrier  to  the  use  of 
the  GSS  was  that  the  UN  body  was  accustomed  to 
flexible  schedules  and  many  of  the  other  meetings  were 
taking  longer  than  was  expected.  The  lessons-learned 
meeting  was  the  first  to  be  canceled  when  the  day’s 
schedule  slipped.  During  the  lessons  learned  sessions 
that  did  take  place  there  was  an  apparent  resistance  to 
using  GSS  by  the  UN  participants.  On  several  occasions 
they  were  observed  moving  the  keyboards  out  of  the  way 
when  sitting  down.  Even  with  instruction,  they  were 
reluctant  to  make  entries  in  the  GSS.  The  military 
participants,  on  the  other  hand,  were  willing  to  use  the 
system  but  did  not  because  of  the  refusal  of  their  UN 
counterparts. 

When  the  UN  and  military  moved  to  a  shore  based 
CMOC,  the  CMI  team  noted  a  marked  change  in  the  UN 
party’s  adherence  to  the  schedule.  The  unfamiliar 
surroundings  of  the  ship  were  gone  and  the  UN  players 
were  in  a  situation  where  their  responsibilities  were 
clear.  UN  members  began  to  engage  and  take  on  the 
challenge  of  teaching  the  military  representatives  the 


finer  points  of  HA/DR  operations.  The  research  team, 
not  yet  aware  of  the  internal  UN  decision  processes, 
remained  puzzled  by  the  UN  staff  s  firm  resistance  to 
the  GSS.  The  UN  users  argued  against  the  use  of  the 
GSS  and  stated  they  felt  the  requirement  to  use  GSS  was 
sprung  upon  them  with  no  warning. 

However,  UN  participants  eventually  explained  the 
reasons  for  not  wanting  to  utilize  the  GSS.  As 
mentioned  before,  they  prefer  to  keep  discussion  private 
until  all  agencies  can  support  a  united  public  front,  and 
they  felt  the  GSS  might  circumvent  their  decision 
process.  The  impression  was  that  recording  individual 
opinions  and  putting  them  before  the  group  could 
undermine  the  consensus  that  the  UN  staff  required.  In 
an  actual  operation,  the  GSS  process  may  have  presented 
a  security  risk  to  the  UN  staff.  Another  factor 
contributing  to  their  reluctance  to  GSS  may  have  been 
the  familiarity  of  the  military  participants  with  the  tool, 
and  a  sense  of  relative  disadvantage  as  new  users.. 

The  first  lessons-learned  session  on  shore  was  filled 
with  tension  due  to  the  disagreement  over  use  of  the 
GSS.  The  research  team,  with  support  from  the  military, 
persuaded  the  UN  group  that  using  the  GSS  could 
benefit  all  exercise  participants.  After  debating  the 
issue,  the  entire  group  decided  to  try  using  the  GSS  on  a 
limited  basis  to  record  lessons  learned.  If  there  were 
future  objections ,  the  GSS  could  be  abandoned. 

After  a  short  GSS  session,  acceptance  of  the 
technology  increased,  and  usage  expanded  throughout 
the  exercise.  By  the  close  of  Strong  Angel,  the  UN 
representatives  were  enthusiastic  about  using  the  GSS  to 
facilitate  their  meetings.  Specifically,  UN  members 
came  to  value  the  speed  and  efficiency  with  which  points 
could  be  brought  up  and  disseminated  to  the  entire 
group. 

Attributes  of  the  GSS  alone  were  not  responsible  for 
the  increased  scope  of  usage  as  the  situation  was 
influenced  by  the  growing  trust  between  the  military  and 
civilian  participants.  Members  of  each  organization 
began  to  identify  with  their  counterparts  and  to  establish 
productive  working  relationships. 

CommandNet  History  &  Use  during  Strong 
Angel  Exercise 

CommandNet,  the  collaborative  log  application  used 
during  the  exercise,  was  developed  as  a  direct  response 
to  Third  Fleet  requirements.  The  Third  Fleet  staff  is 
distributed  through  many  spaces  on  different  decks. 
This  physical  separation  has  caused  difficulties  in 
sharing  information  and  coordinating  effort.  The  Third 
Fleet  staff  had  been  using  GroupSystems  as  a  decision 
and  meeting  support  tool  when  a  Third  Fleet  Intelligence 


officer  had  the  idea  of  using  the  GSS  to  ran  a  24-hour-a- 
day  distributed  meeting.  The  participants  of  this  meeting 
would  be  the  intelligence  collectors  and  analysts 
distributed  throughout  the  ship.  By  meeting  virtually, 
the  group  could  instantly  share  new  intelligence 
information  and  benefit  from  the  group  interaction. 

This  novel  use  of  the  GSS  expanded  to  the 
establishment  of  numerous  collaborative  logs  that  are 
used  for  many  functions  on  the  ship.  There  are  logs  for 
monitoring  intelligence,  exercise  control,  command  and 
control,  battle  watch,  and  others.  CMI’s  experience 
running  the  collaborative  logs  allowed  researchers  to 
recognize  the  limits  of  GroupSystems  as  a  logging  tool. 
GroupSystems  software  had  shortcomings,  such  as 
frequent  network  traffic  and  extensive  client  installation 
requirements. 

To  overcome  these  limitations,  CMI  developed  a 
prototype  Java  application  called  ConmiandNet  that 
specifically  focused  on  logging  functions.  ConmiandNet 
maintains  the  ease  of  use  of  the  GSS  but  utilizes  a  web 
browser  as  a  client.  Additionally,  ConmiandNet  operates 
effectively  under  bandwidth  limitations. 

ConmiandNet  was  introduced  to  Third  Fleet  during 
RIMPAC  2000  and  Strong  Angel.  The  players  in  Strong 
Angel  were  both  civilian  and  military,  so  the  network 
was  the  Internet  rather  than  the  classified  network  used 
for  the  military  logs.  The  collaborative  log  server  was 
set  up  in  California.  Ships  accessed  the  server  through 
standard  satellite  connection  while  the  shore  based 
CMOC  users  accessed  the  logs  through  the  wireless 
LAN’s  Internet  connection.  Logging  started  shortly 
before  the  CMOC  transitioned  ashore  and  continued 
throughout  the  length  of  the  operation. 

Description  of  CommandNet  Research 

MITRE  joined  with  CMI  on  a  research  project  funded 
by  the  Defense  Advanced  Research  Projects  Agency 
(DARPA) 1  to  capture  collaborative  data  and  observe 
human  interactions  in  an  operational  environment. 
Specifically,  the  team  set  out  to  capture  interactions  with 
a  collaborative  logbook  across  groups  of  users  over  time, 
analyze  the  tool  interactions  (including  the  users’  ability 
to  adapt  to  the  system),  and  evaluate  the  effectiveness  of 
the  tool.  The  MITRE  team  brought  experience  in 
instrumentation,  data  capture  (both  automatic  and  via 
observations  and  surveys),  and  analysis  of  tool  usage  and 
collaborative  interaction  (Bayer,  et.  al.,  1999,  Damianos, 
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et.  al.,  1999,  Drury,  et.  al.,  1999,  Finholt  and  Brooks, 
1999,  Kurtz,  et.  al.,  1999,  Damianos,  et.  al.,  2000a, 
Damianos,  et.  al.  2000b). 

There  were  multiple  desirable  outcomes  from  this 
research  project.  Those  outcomes  included  analysis, 
visualization,  and  persistent  history  of  the  captured 
interactions  to  assist  in  event  recreation  and  training  for 
HA/DR,  intelligence  analysis,  and  exercise  control. 
MITRE’s  evaluation  team  provided  iterative  feedback  to 
CMI,  which  was  useful  in  the  development  of  the 
collaborative  tools.  The  ongoing  analysis  of  the 
captured  data  is  leading  to  insights  on  tool  usability  as 
well  as  on  tool  effectiveness  in  supporting  collaborative 
interaction  and  increasing  situational  awareness. 
Additionally,  experiment  results  contribute  to  ongoing 
research  in  evaluation  methodology  and  in  data 
collection  techniques. 

Methodology  for  CommandNet  Research 

The  MITRE  team  collected  both  quantitative  and 
qualitative  data.  Quantitative  data  was  captured  by 
automatic  logging  of  tool  usage  and  through  user 
surveys.  Qualitative  data  was  gathered  through 
participant  observation  and  open-ended  interviews.  This 
combination  of  methods  increased  confidence  in  the 
validity  of  the  results. 

Data  Collection.  Automatic  data  capture  was 
accomplished  in  two  ways.  First,  the  content  of  the  logs 
constituted  an  ongoing  record  of  the  contributions  made, 
the  times  when  they  were  made,  and  the  user  identities 
or  roles  of  the  authors.  In  a  few  isolated  cases,  users 
shared  login  accounts. 

Second,  data  were  automatically  captured  through 
server  logs.  The  server  logs  provided  an  ongoing  event 
record  of  human  interactions  with  CommandNet. 
MITRE  worked  with  CMI  to  instrument  CommandNet 
for  automatic  event  logging.  Data  were  captured  on  the 
server  side  as  users  interacted  with  the  tool.  Various 
types  of  events  were  recorded,  including  navigation  from 
one  screen  to  the  next,  administrative  activity,  and  usage 
of  advanced  features.  Event  logs  allowed  researchers  to 
discern  what  the  user  was  viewing,  when  entries  were 
logged,  and  whether  the  user  searched  or  used  the 
headline  capability  for  analyzing  the  logs.  What  could 
not  be  captured  were  client  side  events  such  as  scrolling 
and  opening  or  closing  of  browser  windows. 

A  questionnaire  was  developed  to  gain  feedback  on 
CommandNet  and  to  collect  background  information  on 
the  participants.  Survey  questions  prompted  participants 
to  compare  CommandNet  to  other  logging  systems  they 
had  used,  to  rate  the  tool's  features,  and  to  comment  on 
what  they  liked  and  did  not  like. 


Background  information  was  collected  on  rank  and 
specialization,  computer  experience,  experience  with 
collaborative  tools,  and  what  media  the  participant  had 
used  prior  to  CommandNet  to  capture  or  access  mission 
critical  information.  Questions  on  overall  experience 
were  designed  to  elicit  feedback  on  usability,  user 
satisfaction  and  ideas  for  tool  improvements. 

The  design  of  the  questionnaire  was  for  either  paper  or 
Internet  administration.  Technical  difficulties  prevented 
the  Internet  version  from  becoming  operational. 
MITRE’s  observer  administered  the  questionnaire 
verbally  and  was  able  to  elicit  extensive  comments  on 
the  open-ended  questions.  Six  out  of  20  participants 
responded  to  survey  questions. 

Open-ended  interviews  were  conducted  to  gather 
contextual  information  about  the  situations  in  which 
CommandNet  was  used.  Sixteen  participants  were 
interviewed,  and  roughly  half  of  the  interviews  were 
audio  taped.  Interviews  took  place  in  formal  sit-down 
interviews  scheduled  in  advance  or  in  impromptu 
conversations  while  waiting  for  meetings  to  begin. 
About  half  of  those  interviews  were  audio  recorded. 
Many  participants,  especially  those  from  the  UN 
agencies,  were  under  considerable  time  pressure  due  to 
the  exigencies  of  the  exercise;  any  time  available  to 
respond  to  questions  was  valuable. 

Interviews  focused  on  how  CommandNet  fit  into  the 
larger  picture  of  what  people  were  doing  during  the 
exercise.  Specific  questions  varied,  depending  on  the 
background  and  role  of  the  participant.  Commonly 
asked  questions  included  how  the  use  of  specific 
CommandNet  logs  was  decided  upon,  and  how  the  tool 
was  introduced  to  the  user.  Enquiries  were  also  made 
about  modifications  participants  would  like  to  see  in 
CommandNet,  and  how  they  envisioned  the  tool  being 
used  in  the  future. 

An  ethnographer  from  MITRE  was  a  participant 
observer  during  Strong  Angel  and  RIMPAC  2000.  Third 
Fleet  and  CMI  initially  granted  the  observer  access  to 
meetings  and  relevant  shipboard  spaces.  She  came  to  be 
considered  a  regular  member  of  the  Strong  Angel 
community  through  participation  in  meetings, 
operations,  and  community  activities.  Daily  observation 
focused  on  how  people  interacted  with  the  collaborative 
technology  and  with  each  other  and  on  organizational 
processes  surrounding  the  use  of  CommandNet. 

Data  Preparation.  Both  quantitative  and  qualitative 
data  were  prepared  to  facilitate  analysis.  Quantitative 
data  from  the  HCI  surveys  were  tabulated  in  an  Excel 
spreadsheet.  CMI  had  instrumented  CommandNet  to 
output  server  log  data  into  XML  documents.  MITRE 
parsed  the  XML  documents  to  create  spreadsheets  for 
manipulation,  table  creation,  and  graphical  displays. 


Qualitative  data  from  surveys  were  organized  by  topic. 
Interviews  and  notes  were  transcribed  and  organized  by 
content.  These  data  were  then  distributed  to  the  MITRE 
research  team,  to  provide  context  for  analyzing  the 
quantitative  data. 

Analysis.  Interpretation  of  observations  and 
interviews  conducted  during  the  exercise  provided 
deeper  insight  into  the  meaning  of  patterns  found  in  the 
server  logs.  Data  from  the  server  logs  were  used  to 
verify  that  user  feedback  reflected  actual  use  of  the  tool. 
Thus,  comparing  data  collected  through  various  methods 
informed  the  team  and  laid  a  solid  foundation  for 
analysis  and  evaluation. 

The  MITRE  research  team  wanted  to  understand  the 
usage  of  CommandNet  during  Strong  Angel  and  whether 
the  tool  features  supported  this  usage  appropriately. 
Contents  of  the  logs  were  annotated  for  a  high-level 


view  of  usage.  Participant-tool  interactions  were 
profiled  over  time  to  look  for  changes  in  behavior  and 
adoption  or  rejection  of  particular  features.  Ease  of  use 
of  CommandNet  was  investigated  particularly  since 
training  had  been  minimal  and  informal. 

Results  of  CommandNet  Research  Project 

Results  of  MITRE's  evaluation  are  divided  into  four 
categories:  characteristics  of  the  user  population,  basic 
usage  patterns,  advanced  feature  usage,  and 
adoption/diffusion.  Below,  selected  highlights  of 
findings  are  discussed. 

User  Population.  The  user  population  for 
CommandNet  during  Strong  Angel  consisted  of  U.S. 
Marines,  UN  team  members,  and  Third  Fleet 
representatives.  All  surveyed  members  had  experience 
in  using  computers  and  had  used  collaborative  tools 
previously  (e.g.,  e-mail,  shared  directories; 
GroupSy stems).  In  the  past,  participants  had  used  a 
variety  of  media  for  reporting  observations  and 


accessing  information  including  email,  paper-based 
logbooks,  and  non- web-based  versions  of  CommandNet. 

Participants  directly  responsible  for  routine 
communications  were  often  the  ones  to  make  entries; 
while  leaders  had  authority  to  make  decisions  about  what 
constituted  appropriate  usage  of  the  tool.  The  tool  was 
relatively  new  to  many,  so  ‘appropriate  usage’  was  not 
always  agreed  upon  or  understood.  Thus, 
adoption/diffusion  was  uneven  across  the  population. 

Basic  Utility.  There  was  general  agreement  that 
CommandNet  was  effective  as  a  common  data 
repository,  making  information  easily  accessible  within 
the  civil  military  context.  Before  CommandNet, 
information  was  gathered,  put  on  a  slip  of  paper,  and 
tacked  to  a  piece  of  wood  nearby.  This  method  often 
failed  to  meet  distribution  needs.  Although  the 
information  had  been  collected,  the  dissemination 
process  lacked  efficacy  and  reliability. 

Eighty  percent  of  the  users  surveyed  indicated 
CommandNet  was  a  useful  tool  while  the  other  20%  did 
not  respond  to  the  question.  Users  were  also  enthusiastic 
about  the  ability  to  scroll  back  through  the  logs  and 
review  data  submitted  by  others. 

Informational  Content.  CommandNet  was  most 
often  used  for  recording  situational  reports  (“SITREP”), 
tracking  resources  and  people,  and  general 
communication  and  coordination.  See  figure  1  for 
content  types  of  entries.  Trouble-shooting  was  another 
common  usage.  In  fact,  CommandNet  turned  out  to  be 
unexpectedly  useful  in  this  regard.  When  the  radios 
intended  for  ship-to-shore  communication  failed, 
CommandNet  was  the  only  working  communication  link 
between  the  CMOC  ashore  and  the  CMOC  afloat.. 

Ease  of  Use.  During  an  initial  humanitarian 
response,  most  people  are  unfamiliar  with  each  other  and 
the  surrounding  context  (i.e.,  unfamiliar  terrain,  politics, 
logistics,  etc.).  Yet  there  is  an  intense  need  to  coordinate 
closely  and  quickly.  Effective  communication  is  critical 


in  this  kind  of  environment,  and  communication  support 
is  a  major  enabler  of  HA/DR  efforts.  The  design  center 
adopted  by  CMI  for  CommandNet  —  “Drop-Dead 
Simple”  -  met  this  critical  need.  One  user  stated 
CommandNet  was  “...very  straightforward,  easy  to  use.” 
CommandNet  being  easy  to  leam  and  use  was  a  very 
significant  advantage  since  many  people  had  to  begin 
work  with  little  or  no  prior  training. 

Fifty  percent  of  the  users  surveyed  found 
CommandNet  easy  to  use;  no  one  found  it  difficult  to  use 
(figure  2).  Sixty-seven  percent  found  CommandNet 
easier  to  use  than  prior  methods.  The  ease-of-use  aspect 
of  the  tool  is  also  highlighted  by  the  fact  that  few  users 
received  formal  training.  Many  users  either  figured  out 
how  to  use  the  tool  themselves  or  learned  by 
demonstration.  Four  survey  participants  were  shown  the 
basics  of  logging  in  and  making  entries,  one  taught 
himself,  and  one  learned  by  watching  another  user. 

Usage  Limits.  With  ease  of  use  and  critical  utility 
both  contributing  to  CommandNet  usage  during  the 
Strong  Angel  exercise,  the  actual  amount  of  usage  was 
lower  than  might  be  expected.  Although  23  user 
accounts  were  created,  just  60%  of  those  users  actually 
made  entries,  and  two  of  those  users  were  CMI 
members.  There  were  105  entries  made  over  7  days. 
Furthermore,  CommandNet  was  not  in  use  in  the  CMOC 
on  ship  before  the  exercise  participants  moved  ashore. 
The  most  plausible  explanation  for  the  minimal  usage 
involves  the  lack  of  advance  planning,  combined  with 
the  urgency  of  other  exercise  activities. 

Advanced  Features.  While  CommandNet’ s  basic 
design  was  "Drop  Dead  Simple”,  it  also  contained  a 
number  of  advanced  features  that  were  not  used  during 
the  exercise.  These  advanced  features  were  generally 
unobtrusive.  Some  (e.g.,  categories  and  importance) 
were  visible,  but  optional,  on  the  main  screen.  Others 
(e.g.,  search,  headlines,  preference  settings)  were 
recessed;  to  access  them,  the  user  would  have  to  click  on 
a  menu  item  that  led  to  a  different  web  page. 

The  most  commonly  used  advanced  feature  was 
categories.  CMI  members  created  10  categories  to  use  in 
the  logs,  but  these  did  not  necessarily  correspond  to 
categories  that  participants  found  useful.  Users 
commented  that  the  category  feature  was  not  used  as 
well  as  it  should  have  been.  If  categories  had  been 
agreed  upon  in  advance  and  their  use  discussed  by  the 
participant  teams,  then  they  might  have  shown  a  higher 
utility  for  log  review  and  monitoring.  Another  easy-to- 
use  feature,  importance,  was  used  very  little  during 
Strong  Angel.  Just  25%  of  all  active  users  accessed  the 
web  page  for  user  preferences,  and  a  mere  1%  of  the 
active  users  navigated  to  the  search  and  headline 
features. 


Advanced  features  were  not  used  for  several  reasons, 
mostly  related  to  the  short  duration  of  the  exercise  under 
‘emergency’  conditions  without  much  lead  planning. 
Had  the  tool  been  in  use  for  a  longer  time,  there  might 
have  been  more  exploration  and  experimentation  with 
different  features,  which  may  or  may  not  have  been 
adopted  by  the  general  population. 

Adoption  and  Diffusion 

As  suggested  above,  there  were  conflicting  influences 
on  the  adoption  and  diffusion  of  CommandNet 
throughout  the  Strong  Angel  population.  MITRE's 
evaluation  efforts  explored  how  the  influences  affected 
tool  adoption  and  diffusion  and  led  to  variations  in  usage 
across  different  sub-populations.  In  addition  to  ease-of- 
use  and  planning,  other  influences  include  (non-) 
mobility  of  participant's  role  in  the  exercise,  effect  of 
urgency  of  HA/DR  contingencies  on  leadership,  and 
access  to  CMI  support  personnel. 

The  matter  of  relative  mobility  is  significant.  Some 
people  (especially  those  responsible  for  routine 
communications)  had  jobs  that  kept  them  in  a  single 
location  throughout  the  day.  Others  had  jobs  that 
involved  routine  information  reporting,  and  they  were 
the  ones  most  likely  to  use  the  tool.  Still  other 
participants  were  involved  in  the  operations  of  setting  up 
a  camp  and  roaming  the  site  in  order  to  address  a  wide 
range  of  issues.  Such  people  did  not  have  ongoing 
access  to  a  networked  computer  that  would  readily 
afford  them  opportunities  to  access  the  collaborative 
log(s).  For  those  participants,  the  potential  usefulness  of 
wireless  communications  was  fairly  clear. 

To  put  the  tool  to  optimal  use  for  HA/DR  processes 
would  have  involved  a  certain  degree  of  advance 
planning  -  i.e.,  deciding  how  SITREPs  should  be 
prepared  for  posting,  agreeing  on  what  kinds  of  tracking 
to  use  the  tool  for,  etc.  Such  preparations  had  not  been 
made  in  advance  in  this  case,  so  the  usage  of  the  tool 
was  limited  by  the  fact  that  there  was  little  consensus  on 
how  to  use  the  tool.  Additionally,  there  were  few  leaders 
available  to  provide  support  and  reinforcement  in  that 
regard.  Having  just  landed  in  unfamiliar  terrain,  and 
facing  the  imminent  arrival  of  thousands  of  refugees, 
HA/DR  leaders  needed  to  be  much  more  concerned  with 
securing  food,  water  and  shelter  for  large  numbers  of 
people  than  focused  on  the  learning  of  a  new 
collaborative  tool. 

Another  factor  influencing  adoption  and  diffusion  was 
access  to  the  developers  and  related  support  personnel. 
While  one  computer -savvy  individual  took  it  upon 
himself  to  explore  and  experiment  with  CommandNet, 
others  needed  more  support  to  feel  comfortable  using  the 
tool.  The  most  frequent  users  of  CommandNet  during 


Strong  Angel  turned  out  to  be  those  people  who  were 
stationed  in  the  same  location  as  the  CMI  support 
personnel.  As  a  result  of  this  close  proximity,  they  could 
easily  ask  impromptu  questions  and  get  immediate 
assistance. 

Discussion  of  Results  from  a  Collaborative 
Technologies  Perspective 

These  results  are  consistent  with  current  themes  in 
development  of  collaborative  technologies:  (a) 

alignment  of  ‘fit"  between  technologies  and  the 
mission/culture  of  the  user  organization,  (b)  variation  in 
user  behavior  within  and  across  tasks  and  user 
communities,  and  (c)  need  for  assessment  within  a 
mission  context. 

Successful  transition  of  technologies  into  operational 
environments  requires  organizational  commitment,  as 
well  as  clear  requirements  and  good  alignment  between 
the  technologies  and  tasks  to  be  perfomred.  This 
research  shows  the  military  environment  is  an  excellent 
testbed  for  assessment  and  transition  of  collaboration 
technologies.  The  requirement  to  collaborate  is  clear, 
missions  are  well-defined  and  the  participants  are 
knowledgeable,  share  doctrine  and  are  highly  motivated 
to  succeed 

In  Strong  Angel,  CommandNet  was  not  only  easy  to 
use,  but  it  directly  supported  a  function  structurally 
integral  to  naval  operations:  the  keeping  of  logs  and 
situation  reports.  Military  users  knew  how  to  compose 
and  maintain  logs  and  SITREPS  and  how  to  share  them. 
However,  use  of  CommandNet  was  not  limited  to 
logging,  and  its  utility  as  a  more  general 
communications  medium  became  both  visible  and 
valuable  when  radio  communications  could  not  be 
established  between  ship  and  shore,  and  CommandNet 
became  the  only  electronic  way  of  exchanging  messages. 
Users  may  begin  to  use  a  tool  for  specific  purposes,  but 
they  evolve  their  usage  and  also  their  requirements  into 
areas  not  within  the  original  design  center  of  the  tool. 
This  highlights  the  need  for  analysis  of  collaborative 
interaction  over  time,  as  users  become  familiar  with 
collaborative  tools  and  use  them  in  broader  aspects  of 
their  work  environment. 

Instrumented  data  capture,  complemented  by 
interviews,  questionnaires  and  other  observations,  is 
essential  to  reflect  both  quantitative  usage  patterns  and 
the  texture  of  the  social  communication  that  underlies 
collaboration. 

Impact  of  collaboration  technology  on  a 
combined  Military  &  UN  HA/DR  exercise 


The  introduction  of  GroupSystems  as  a  collaborative 
tool  in  Strong  Angel  came  as  a  surprise  to  many  of  the 
non-military  participants.  Despite  discussions  during  the 
planning  meetings,  the  United  Nations  agency 
representatives  found  themselves  on  new  ground  during 
the  initial  days  of  the  exercise,  and  their  hesitation  was 
readily  apparent.  Efforts  were  made  to  reassure 
participants  that  the  material  submitted  each  evening  was 
only  for  improving  interactions,  not  for  keeping  an 
electronic  record.  Members  of  the  military  leadership 
respected  the  caution  of  the  UN  participants  but 
collaborative  technology  usage  was  encouraged. 

In  a  short  time  the  collaboration  technology  came  to 
serve  as  a  catalyst  for  bilateral  civil-military  education. 
Using  GroupSystems,  the  participants  found  they  could 
submit  a  thought  anonymously,  watch  the  reaction,  guide 
the  discussion  in  directions  they  found  appropriate  and 
informative,  then  drop  back  quietly  as  the  group  came  to 
a  consensus  on  that  issue.  Non-military  agencies  work 
mostly  by  consensus,  different  from  the  hierarchical 
military  approach.  The  participants  found 

GroupSystems,  which  is  designed  to  optimize 
consensus-based  decision  support,  helped  the  military 
more  effectively  work  in  a  consensus  decision-making 
mode. 

In  addition,  the  GroupSystems  technology  helped 
focus  the  group  on  the  particular  goals  of  the  exercise. 
Each  night  the  senior  and  middle  leadership  gathered 
around  the  table  to  think  constructively  and 
collaborative  ly  about  the  day.  Since  each  day  contained 
innumerable  examples  of  differing  civilian  and  military 
approaches  ,  the  evening  meeting  allowed  each  facet  of 
the  exercise  to  be  evaluated  cooperatively.  Then  each 
member  of  the  group  was  able  to  extract  the  larger  and 
more  meaningful  lessons  of  the  day.  The  computer- 
supported  collaboration  sessions  also  helped  participants 
realize  that  these  few  hours  in  the  evening  must  be  an 
on-going  part  of  the  process.  Collaborative  dscussion 
reinforced  the  significance  of  combined  participation  in 
maintaining  the  humanitarian  focus. 

For  the  evening  meeting,  each  entry  was  reviewed  by 
the  group  as  a  whole,  rephrased  by  consensus  as 
appropriate,  then  listed  under  a  topic  area  and  filed  for 
later  consolidation.  The  result  was  effectively  a  journal 
of  the  civil-military  consolidation  process,  marking  a 
path  through  the  minefields  of  civil-military  cooperation. 
That  collaborative  effort  each  evening  was  the  one 
constant  in  an  exercise  that  was  forced  to  flex  and  adapt 
across  every  conceivable  impediment,  from  dust  storms 
to  death  threats,  from  having  the  water  supply  severed  by 
an  auto  accident  to  having  an  access  road  declared 
unusable,  and  finally  finding  WWII  bombs  within  the 
refugee  camp  itself.  The  UN  and  military  reviewed  the 
interaction  each  evening  and  planned  both  to  solve  the 


immediate  problems  and  to  develop  management  tools 
for  such  events  in  the  future. 

A  frequent  refrain  in  any  civil-military  operation  is 
that  there  must  be  a  transition  from  military  support  to 
civilian  support  just  as  rapidly  as  possible.  That 
transition  allows  both  for  the  military  to  go  home  and  for 
the  continuation  of  national  will  to  accept  military 
participation.  One  result  of  the  collaboration  technology 
in  Strong  Angel  was  a  template  for  the  transition 
developed  through  a  cooperative  effort  between  the  UN 
agency  representatives  and  the  U.S.  military,  particularly 
the  Marine  Corps.  Participants  stated  it  was  unlikely 
such  an  effort  could  have  been  so  productive  without  the 
collaborative  practice  forged  by  GroupSystems. 

Although  the  collaboration  techniques  developed 
throughout  Strong  Angel  were  far  from  perfect,  everyone 
learned  a  great  deal  about  where  software  like 
GroupSystems  is  more  and  less  valuable.  It  is  not  easy, 
for  example,  to  address  issues  that  require  a  sequential 
conversation  or  significant  depth.  Technologies  such  as 
GroupSystems  are  not  designed  for  those  purposes.  The 
military  and  the  UN  took  other  opportunities  to  sit  down 
and  discuss  the  philosophical  differences  between  them 
and  to  look  at  future  opportunities  for  the  evolution  of 
what  appeared  to  be  a  useful  exercise.  Participants  soon 
saw,  though,  that  the  fruits  of  those  discussions  were 
reflected  in  the  topics  addressed  later  that  evening  as  the 
lessons -learned  were  captured  on  GroupSystems  for  the 
day. 

In  sum,  collaboration  technology  served  as  a  tool  for 
introducing  disparate  groups  to  each  other,  and  it 
accelerated  the  productive  work  that  each  group  needed 
to  accomplish.  Such  a  resource  holds  much  promise  for 
improving  both  exercises  and  real-world  operations  that 
cross  civil-military  boundaries.  Significantly,  a  senior 
UN  representative  stated  that,  "accelerated  integration 
improves  the  rate  of  real  lives  saved,  and  we  agree  that 
all  proven  tools  should  be  incorporated  routinely.” 
Several  plans  for  future  civil-military  exercises  and 
operations  now  incorporate  collaboration  fechnology  as 
one  mechanism  for  doing  a  better  job. 

Future  Directions 

A  multitude  of  factors  made  collaboration  difficult 
during  Strong  Angel.  First,  collaboration  between 
organizations  whose  missions  are  both  different  and 
unfamiliar  is  demanding.  Second,  past  interactions 
between  some  organizations  have  created  actual  and 
perceived  strains  and  minors  regarding  how  each  does 
business.  Third,  participants  came  from  different 
cultural  backgrounds  and  used  multiple  languages  to 
communicate.  Fourth,  the  environmental  conditions 


were  extreme,  and  the  pressure  of  making  critical  life 
threatening  decisions  under  time  constraints  added  to  the 
difficulty  associated  with  collaborative  decision-making. 

These  difficulties  were  balanced  by  the  strong 
motivation  of  participants  to  perform  their  mission  well. 
Because  the  participants  in  Strong  Angel  came  to  the 
exercise  with  extensive  experience  in  actual 
humanitarian  assistance  efforts  and  knew  first  hand  the 
human  consequences  of  ineffective  collaboration. .  They 
found  the  opportunity  to  explore  collaborative 
techniques  genuinely  worthwhile. 

One  lesson  learned  is  that  humanitarian  organizations 
will  usually  sacrifice  geographic  proximity,  and  trade-off 
travel  time  to  the  refugee  site,  in  order  to  provide 
reasonably  comfortable,  secure  working  conditions  for 
their  staffs,  information  technology,  and 
communications  equipment.  Not  only  does  this 
minimize  wear  and  tear  on  equipment,  but  it  also 
alleviates  a  significant  degree  of  stress  on  the 
humanitarian  workers,  enabling  them  to  work  more 
effectively.  The  dust,  wind,  rain  and  extremes  of 
temperature  exacted  a  significant  toll  on  the  individuals 
working  in  the  camp.  Relocation  to  a  secure, 
environmentally  protected  workspace  would  be  an 
extremely  high  priority  for  humanitarian  organizations 
confronted  with  the  conditions  encountered  during 
Strong  Angel. 

This  study  was  instrumental  in  shaping  CMI’s 
thinking  on  how  to  develop  collaborative  technology  and 
processes  to  support  decision-making  under  the 
constraints  of  humanitarian  assistance  efforts.  One 
future  research  direction  is  to  develop  processes  to 
integrate  collaborative  technology  in  a  self-sustaining 
community  within  the  infrastructure  of  a  humanitarian 
assistance  effort.  Briggs,  et.  al.  (1999)  suggest  focusing 
on  identifying  and  supporting  repeatable  processes  early 
in  the  event  to  promote  the  transition  of  a  technology 
within  an  organization. 

A  second  research  direction  is  to  identify  collaborative 
requirements  in  humanitarian  assistance  efforts  and  to 
modify  CommandNet  to  increase  situational  awareness 
among  participants  in  an  operation.  The  objective  of 
CommandNet  is  to  improve  communication  and  shorten 
the  decision  cycle  by  making  collaboration  and 
information  sharing  routine.  CommandNet  could 
constitute  an  operational  information  clearinghouse, 
improving  situational  awareness  for  all  participants 
during  humanitarian  support  operations. 

A  third  research  direction  is  to  focus  on  the  lessons- 
leamed  from  working  in  a  CMOC  afloat  and 
transitioning  to  facilities  ashore.  Research  on  using 
collaborative  technology  in  portable  self-sustaining  units 
that  function  in  various  austere  environments  is  required 
to  improve  the  capabilities  of  Civil  Military  Operations 


Centers.  Finally,  it  is  crucial  to  continue  research  on 
assessment  of  collaborative  technologies  in  operational 
environments,  collecting  both  quantitative  and 
qualitative  data  about  user  interactions  and  judgments  on 
tool  use. 
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